412TW-PA-15214 


HANDLING  QUALITIES  EVALUATIONS  OF 
UNMANNED  AIRCRAFT  SYSTEMS 


Christopher  Baughman,  Maj,  USAF 
Kevin  Longeuay,  GA-ASI 


AIR  FORCE  TEST  CENTER 
EDWARDS  AFB,  CA 

4  May  2015 


Approved  for  public  release  ;  distribution  is  unlimited. 
412TW-PA-15214 


AIR  FORCE  TEST  CENTER 
EDWARDS  AIR  FORCE  BASE,  CALIFORNIA 
AIR  FORCE  MATERIEL  COMMAND 
UNITED  STATES  AIR  FORCE 


REPORT  DOCUMENTATION  PAGE 


Form  Approved 
OMB  No.  0704-0188 


Public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and  maintaining  the 
data  needed,  and  completing  and  reviewing  this  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information,  including  suggestions  for  reducing 
this  burden  to  Department  of  Defense,  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports  (0704-0188),  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington,  VA  22202- 
4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  any  penalty  for  failing  to  comply  with  a  collection  of  information  if  it  does  not  display  a  currently 
valid  OMB  control  number.  PLEASE  DO  NOT  RETURN  YOUR  FORM  TO  THE  ABOVE  ADDRESS. 


1.  REPORT  DATE  (DD-MM-YYYY)  2.  REPORT  TYPE  3.  DATES  COVERED  (From  -  To) 

04-05-2015  Technical  Paper  March  2014-May  2015 


4.  TITLE  AND  SUBTITLE  5a.  CONTRACT  NUMBER 


HANDLING  QUALITIES  EVALUATIONS  OF  UNMANNED  AIRCRAFT  SYSTEMS 


5b.  GRANT  NUMBER 


5c.  PROGRAM  ELEMENT  NUMBER 


6.  AUTHOR(S) 

Christopher  Baughman,  Maj,  USAF 
Kevin  Longeuay,  CNTR,  GA-ASI 


5d.  PROJECT  NUMBER 


5e.  TASK  NUMBER 


5f.  WORK  UNIT  NUMBER 


7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES)  AND  ADDRESS(ES) 

USAF  Test  Pilot  School 
220  S.  Wolfe  Ave,  Bldg  1220 
Edwards,  CA  93524 


8.  PERFORMING  ORGANIZATION  REPORT 
NUMBER 

412TW-PA-15214 


9.  SPONSORING  /  MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 


10.  SPONSOR/MONITOR’S  ACRONYM(S) 

N/A 


11.  SPONSOR/MONITOR’S  REPORT 
NUMBER(S) 


12.  DISTRIBUTION  /  AVAILABILITY  STATEMENT 

Approved  for  public  release  A:  distribution  is  unlimited. 


13.  SUPPLEMENTARY  NOTES 

CA:  Air  Force  Test  Center  Edwards  AFB  CA 


CC:  012100 


14.  ABSTRACT 

Since  1969,  the  Cooper-Harper  Rating  Scale  has  been  used  widely  in  flight  test  to  qualitatively  evaluate  the  handling  qualities  of  manned 
aircraft.  Within  the  past  decade,  the  use  and  popularity  of  Unmanned  Aircraft  Systems  (UAS)  has  expanded  into  military  operations  and 
commercial  applications.  Therefore,  there  is  a  need  to  understand  the  handling  qualities  of  these  systems  when  a  pilot  is  required  for  in-the- 
loop  control.  Key  differences  in  flight  control  systems,  autonomous  flight  capabilities,  and  overall  system  latency  in  unmanned  versus 
manned  aircraft  necessitate  the  development  of  approaches  to  enhance  understanding  of  these  systems.  This  paper  identifies  the  need  for 
Cooper-Harper  evaluations  of  UAS,  highlights  the  differences  between  manned  and  unmanned  systems,  and  provides  an  approach  to 
determining  the  Cooper-Harper  evaluation  of  unmanned  systems. 


15.  SUBJECT  TERMS 

UAS,  RPA,  handling  qualities,  Cooper-Harper,  PIO,  boundary  avoidance  tracking 


16.  SECURITY  CLASSIFICATION  OF: 

Unclassified 

a.  REPORT 

Unclassified 

b.  ABSTRACT 

Unclassified 

c.  THIS  PAGE 

Unclassified 

17.  LIMITATION 

18.  NUMBER 

OF  ABSTRACT 

OF  PAGES 

None 

23 

19a.  NAME  OF  RESPONSIBLE  PERSON 

412  TENG/EN  (Tech  Pubs) 


19b.  TELEPHONE  NUMBER  (include  area 
code) 

661-277-8615 


Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std.  Z39.18 


US15-Longeuay 


HANDLING  QUALITIES  EVALUATIONS  OF  UNMANNED 

AIRCRAFT  SYSTEMS 

Kevin  Longeuay*  and  Christopher  Baughman1^ 


Since  1969,  the  Cooper-Harper  Rating  Scale  has  been  used  widely  in  flight  test 
to  qualitatively  evaluate  the  handling  qualities  of  manned  aircraft.  Within  the 
past  decade,  the  use  and  popularity  of  Unmanned  Aircraft  Systems  (UAS)  has 
expanded  into  military  operations  and  commercial  applications.  Therefore,  there 
is  a  need  to  understand  the  handling  qualities  of  these  systems  when  a  pilot  is 
required  for  in-the-loop  control.  Key  differences  in  flight  control  systems,  au¬ 
tonomous  flight  capabilities,  and  overall  system  latency  in  unmanned  versus 
manned  aircraft  necessitate  the  development  of  approaches  to  enhance  under¬ 
standing  of  these  systems.  This  paper  identifies  the  need  for  Cooper-Harper 
evaluations  of  UAS,  highlights  the  differences  between  manned  and  unmanned 
systems,  and  provides  an  approach  to  determining  the  Cooper-Harper  evaluation 
of  unmanned  systems. 

INTRODUCTION 

Evaluating  Handling  Qualities  of  an  Aircraft 

In  aircraft  design,  engineers  need  to  consider  both  the  unaugmented  flying  qualities  of  the  air¬ 
craft  and  the  handling  qualities  of  the  aircraft  with  a  pilot  in  the  loop.  An  aircraft  can  be  very  sta¬ 
ble  in  all  axes  of  flight  at  nominal  conditions,  but  without  proper  design  of  handling  qualities,  the 
pilot  may  have  difficulty  controlling  the  aircraft  in  either  normal  operations  or  in  “comer  cases,” 
which  are  usually  only  found  via  a  thorough  handling  qualities  evaluation  during  flight  test. 
These  instances  are  often  referred  to  as  handling  qualities  “cliffs”  in  which  aircraft  control  ap¬ 
pears  normal  until  a  condition  in  which  rapid  degradation  occurs.  Best  described  by  George 
Cooper  and  Robert  Harper,  handling  qualities  are  defined  as  “those  qualities  or  characteristics  of 
an  aircraft  that  govern  the  ease  and  precision  with  which  a  pilot  is  able  to  perform  the  tasks  re¬ 
quired  in  support  of  an  aircraft  role.”1  Therefore,  it  is  important  to  know  the  role  of  the  aircraft  in 
order  to  properly  design  the  Pilot  Vehicle  Interface  (PV1)  to  support  those  roles.  Table  1  below 
provides  some  aircraft  roles  and  common  tasks  associated  with  those  roles. 


Table  1  Common  Aircraft  Roles  and  Operational  Tasks 


Tasks  \  Roles 

Fighter/Strike 

Bomber 

Surveillance 

Taxi 

X 

X 

X 

Takeoff 

X 

X 

X 

Engineer,  System  Test  &  Qualification,  General  Atomics  ASI,  12385  First  American  Way,  Poway,  CA  92064. 
'  Instructor  Test  Pilot,  USAF  Test  Pilot  School,  Edwards,  CA  93524. 
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Tasks  \  Roles 

Fighter/Strike 

Bomber 

Surveillance 

Climb 

X 

X 

X 

Formation  Flight 

X 

X 

Precision  Strike 

X 

X 

Air-to-Air  Combat 

X 

Aerial  Refueling 

X 

X 

Surveillance 

X 

X 

X 

Fanding 

X 

X 

X 

Every  aircraft  needs  to  be  able  to  complete  takeoff,  climb,  and  landing.  Additional  tasks  may 
need  to  be  accomplished  depending  on  the  aircraft’s  mission.  From  a  handling  qualities  perspec¬ 
tive,  the  tasks  that  require  the  tightest  control  of  the  aircraft  and  are  the  most  stressful  on  the  pilot 
are  generally  the  most  difficult  for  the  pilot  to  perform.  When  coupled  with  critical  phases  of 
flight,  close  proximity  to  the  ground  or  other  aircraft,  or  releasing  ordinance,  the  need  for  design¬ 
ing  in  desirable  handling  qualities  is  essential. 

As  an  operator  of  the  aircraft,  the  pilot  is  a  part  of  the  control  loop.  The  pilot’s  involvement  in 
the  control  loop  is  represented  in  the  block  diagram  in  Figure  l2. 


Handling  Qualities  Control  Loop  -  Manned  Platform 


o 


Figure  1  Handling  Qualities  Control  Loop  -  Manned  Platform 

In  a  manned  platform,  the  pilot  uses  aural,  visual,  and  motion  cues  to  provide  an  input  to  the 
system  for  a  given  task.  Aural  cues  include  engine  response,  warning  indications,  and  aerodynam¬ 
ic  acoustics  such  as  buffeting  which  provides  feedback  on  aerodynamic  response  of  the  aircraft. 
Visual  cues  include  aircraft  position  relative  to  the  horizon  using  forward  and  peripheral  vision, 
critical  aircraft  parameters  displayed  on  a  Heads  Up  Display  (HUD)  or  Heads  Down  Display 
(HDD),  navigation  referencing,  and  spatial  referencing  (for  formation,  air-to-air  combat,  and  aer¬ 
ial  refueling).  Motion  cues  include  all  the  forces  the  pilot  is  subjected  due  during  maneuvering  of 
the  aircraft.  These  motion  cues,  or  proprioceptive  feedback,  allow  the  pilot  to  “feel”  the  aircraft 
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response  and  apply  corrective  inputs  accordingly.  The  input  receiver  on  an  aircraft  traditionally 
consists  of  a  single  axis  throttle  lever  to  control  power  from  the  propulsion  system,  a  two  axis 
control  stick  to  control  pitch  and  roll,  and  a  dual  axis  pedal  system  to  control  yaw  and  braking. 
Using  these  various  sensory  cues  and  the  input  receiver,  the  pilot  can  fine-tune  the  desired  re¬ 
sponse  of  the  aircraft  to  complete  the  task  at  hand. 

Testing  of  handling  qualities  is  widely  performed  using  the  Cooper-Harper  rating  scale.  Es¬ 
tablished  in  1969,  the  Cooper-Harper  rating  system  evaluates  handling  qualities  of  an  aircraft  sys¬ 
tem  by  having  the  pilot  perform  operationally  representative  tasks  within  specific  desired  and  ad¬ 
equate  performance  criteria  and  then  providing  feedback  on  the  performance  attained,  the  work¬ 
load  required,  and  amount  of  pilot  compensation  needed  to  perform  the  task3.  This  system  allows 
the  engineers  to  determine  the  handling  qualities  allow  the  pilot  to  achieve  the  operational  tasks 
which  the  aircraft  was  designed  for.  After  performing  an  operational  task,  the  pilot  uses  the 
Cooper-Harper  rating  scale,  shown  in  Figure  2.  The  scale  provides  a  means  to  measure  task  per¬ 
formance  and  assess  workload,  which  drives  overall  rating. 

The  results  of  a  handling  qualities  evaluation  are  important  in  determining  how  well  the  air¬ 
craft  can  be  controlled  by  the  pilot.  Cooper-Harper  Ratings  (CHR)  are  given  numerically  from  1 
to  10,  with  Level  ratings  from  1  to  3.  A  Level  1  aircraft  is  described  by  a  CHR  of  1-3,  and  is  con¬ 
sidered  to  have  desirable  handling  qualities  with  no  deficiencies.  A  Level  2  aircraft  will  have  a 
CHR  of  4-6,  with  minor  deficiencies  that  warrant  improvement.  In  Level  2  aircraft,  pilot  compen¬ 
sation  is  generally  relied  upon  to  maintain  safety  and  increase  pilot  performance,  however  this 
comes  at  a  price  of  additional  training  and  increased  risk.  A  Level  3  aircraft  (CHR  7-9)  has  signif¬ 
icant  deficiencies,  in  which  improvements  are  required  and  control  is  either  borderline  or  very 
difficult.  A  CHR  of  10  indicates  that  the  aircraft  is  uncontrollable.  Most  military  aircraft  exhibit 
Level  1  handling  qualities  and  some  have  Level  2  tendencies.  For  example,  the  C-17  exhibited 
some  Level  2  tendencies  in  the  roll  axis,  even  after  a  roll  Pilot-in-the-loop  Oscillation  (PIO)  prob¬ 
lem  was  mitigated4.  In  some  cases,  an  aircraft  can  have  Level  2  or  Level  3  handling  qualities  for  a 
specific  task.  For  the  parachute  extraction  mission  at  low  altitude,  the  C-130  demands  the  pilot  fly 
precisely  while  in  close  proximity  to  terrain.  According  to  one  U.S.  Air  Force  test  pilot,  the  C-130 
is  “Level  2  at  best,  mostly  Level  3.”5  In  a  UAS,  Level  1  handling  qualities  are  even  more  elusive 
due  to  the  effects  of  time  delay  and  lack  of  proprioceptive  feedback.  This  will  be  discussed  later 
in  the  UAS  datalink  section. 
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AIRCRAFT 

CHARACTERISTICS 

DEMANDS  ON  THE  PILOT 

IN  SELECTED  TASK  OR  REQUIRED  OPERATION  - 

|  Excellent 

Pilot  compensation  not  a  factor  for 

1  Highly  desirable 

desired  performance 

J  Good 

Pilot  compensation  not  a  factor  for 

|  Negligible  deficiencies 

desired  performance 

Fair  —  Some  mildly 

Minimal  pilot  compensation  required  for 

unpleasant  deficiencies 

desired  performance 

ADEQUACY  FOR  SELECTED  TASK  OR 
REQUIRED  OPERATION* 


Major  deficiencies 


Control  will  be  lost  during  some  portion  ot 
required  operation 


Minor  but  annoying 

Desired  performance  requires  moderate 

deficiencies 

pilot  compensation 

Moderately  objectionable 

Adequate  performance  requires 

deficiencies 

considerable  pilot  compensation 

Very  objectionable  but 

AdoqiialG  performance  requires  extensive 

tolerable  deficiencies 

pilot  compensation 

Major  deficiencies 

Adequate  performance  not  attainable  with 
maximum  tolerable  pilot  compensation. 
Controllability  not  in  question 

Major  deficiencies 

Considerable  pitot  compensation  is  required 
for  control 

Major  deficiencies 

Intense  pitot  compensation  is  required  to 
retain  control 

Cooper-Harper  Ref  NASA  TND-5153 


‘  Definition  of  required  operation  involves  designation  of  flight  phase 
and/or  subphases  with  accompanying  conditions 


Figure  2  Cooper-Harper  Rating  Scale 

To  execute  handling  qualities  testing,  the  test  team  needs  to  first  determine  the  operational 
tasks  required  of  the  aircraft  in  order  to  fulfill  its  role  in  the  mission.  These  operational  tasks 
should  be  any  tasks  required  by  the  pilot  in  order  to  achieve  a  given  mission.  For  each  task,  ade¬ 
quate  and  desired  performance  criteria  are  created  based  on  mission  specific  performance  charac¬ 
teristics  for  the  system  under  test.  An  example  of  performance  criteria  for  an  unmanned  vehicle 
during  a  normal  takeoff  is  listed  below  in  Table  2: 


Table  2  Example  Takeoff  Performance  Criteria 


Measure  of  Performance 

Desired  Criteria 

Adequate  Criteria 

Pitch  attitude  at  rotation 

5°  ±0.5° 

5°  ±2° 

Airspeed 

V climb  ±  2  K1AS 

Vclimb  ±  3  KIAS 

Heading  Control 

Runway  Track  ±  2° 

Runway  Track  ±5° 

No  observed  roll  /yaw 
oscillations 

No  observed  roll/yaw 
oscillations 

For  this  task,  the  pilot  will  attempt  to  pitch  the  aircraft  to  +5°  initially  and  adjust/maintain 
pitch  attitude  to  maintain  a  target  climb  speed  ±2  K1AS  during  the  initial  climb  out.  This  task  is 
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appropriate  especially  for  an  aircraft  with  an  attitude-command  flight  control  system,  which  will 
be  discussed  in  depth  later.  The  pilot  will  also  attempt  to  maintain  aircraft  tracking  on  runway 
centerline  through  the  takeoff  roll  until  after  rotation  and  initial  turn  out.  After  the  task  has  com¬ 
pleted,  the  pilot  will  determine  the  workload  required  for  the  performance  attained  and  will  pro¬ 
vide  any  other  comments  relating  to  the  task.  The  pilot  might  comment  on  how  much  force  on  the 
control  stick  was  required,  if  there  was  a  lot  dead  band  in  the  stick,  if  the  breakout  force  to  move 
the  stick  from  neutral  seemed  high,  if  applying  a  pitch  input  resulted  in  roll  inputs  as  well,  control 
harmony,  stick  sensitivity,  or  any  other  applicable  factors.  When  determining  workload,  the  pilot 
will  consider  any  environmental  factors  that  affected  performance  and  increased  pilot  workload. 
Examples  of  environmental  factors  are  high  crosswind  conditions,  turbulence,  extremely  high 
temperatures,  moisture,  or  standing  water  on  the  runway.  Additionally,  pilot  compensation  must 
be  taken  into  consideration.  Pilot  compensation  occurs  in  many  forms,  both  mentally  and  physi¬ 
cally.  Some  examples  of  this  are  flying  a  maneuver  with  low  gains  to  try  and  stay  out  of  the  loop 
with  the  aircraft,  gripping  the  control  stick  a  certain  way,  and  “leading”  control  inputs  based  on 
the  pilot’s  mental  model  of  aircraft  behavior.  Pilot  compensation  is  probably  the  most  difficult 
parameter  to  evaluate  as  it  requires  the  pilot  to  carefully  consider  and  separate  the  effects  of  how 
and  why  the  aircraft  behaved  in  the  manner  in  which  it  did.  Additionally,  the  difference  between 
minimal  and  moderate  compensation  may  mean  the  difference  between  a  Level  1  and  Level  2 
aircraft.  If  not  properly  evaluated,  a  Level  2  aircraft  may  be  deemed  Level  1  by  missing  key  fac¬ 
tors  of  pilot  compensation. 

When  determining  the  Cooper-Harper  rating  for  the  task,  it  is  important  that  the  pilot  maintain 
the  proper  sequencing  of  the  Cooper-Harper  scale  and  not  rely  solely  on  the  individual  rating  de¬ 
scription.  Also,  it  is  important  for  the  pilot  to  rate  the  task  immediately  after  it  is  performed.  Any 
later  changes  to  the  pilot  ratings  are  discouraged  since  the  reason  for  the  change  may  be  biased. 
More  information  for  proper  use  of  the  Cooper-Harper  rating  scale  can  be  found  in  Cooper  and 
Harper’s  original  report,  written  in  19696. 

In  addition  to  Cooper-Harper  ratings,  the  pilots  should  be  encouraged  to  provide  comments  on 
the  handling  qualities  of  aircraft  during  the  task.  These  comments  should  be  collected  as  an  addi¬ 
tion  to  the  ratings  but  should  be  in  simple  language  (i.e.,  should  not  be  translated  into  engineering 
terms  or  aircraft  responses)  and  should  focus  on  describing  the  difficulty  in  performing  specific 
inputs  and  how  that  difficulty  may  or  may  not  affect  overall  task  performance. 

Since  the  Cooper-Harper  ratings  are  qualitative  in  nature,  it  is  important  that  each  test  be  re¬ 
peated  using  multiple  pilots.  Using  multiple  pilots  provides  a  range  of  evaluations  and  some  pi¬ 
lots  may  identify  poor  areas  of  handling  qualities  that  were  overlooked  by  other  pilots. 

UAS  Expansion  in  Military  Operations 

UAS  fall  into  five  groups,  from  small  UAS  (sUAS)  to  larger  Remotely-Piloted  Aircraft  (RPA) 
that  fly  at  higher  altitudes,  faster  speeds,  and  have  longer  ranges.  Figure  3  below  outlines  Groups 
1-5. 
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Raven  (RQ-11) 


21-55 

<  3.500  AGL 

<  250 

Scan  Eagle 

<  1.320 

<  FL  180 

<250 

Currently  No  USAF 
Program  of  Record 
for  thts  Category 

>  1.320 

<  FL  180 

Any  Airspeed 

Predator  (MO- 1) 

►  FL  180  Any  Airspeed 


Reaper  (MO-9) 


Global  Hawk  (RQ-4) 


Figure  3  Groups  of  UAS7 

Generally,  Groups  4  and  5  are  considered  RPA,  while  Groups  1-3  are  categorized  as  small 
UAS,  or  sUAS.  UAS  have  been  developed  and  used  since  World  War  II,  and  although  at  face 
value  they  may  be  regarded  as  newcomers  to  flight  test  circles,  only  recently  has  the  industry  ex¬ 
perienced  broad  growth.  The  Association  for  Unmanned  Vehicle  Systems  International,  or 
AUVSI,  recognizes  715  companies  which  produce  over  2400  platforms*.  Among  these  systems,  a 
majority,  especially  those  in  Groups  4-5  employ  higher  levels  of  autonomy  -  some  almost  exclu¬ 
sively  -  in  order  to  accomplish  their  missions. 

The  Air  Force  has  put  a  great  deal  of  thought  into  not  only  categorizing  UAS  and  RPA,  but  al¬ 
so  in  defining  its  vision  for  the  next  25  years  with  regard  to  integrating  unmanned  vehicles  into 
airpower  doctrine.  On  the  other  hand,  very  little  has  been  done  to  define  autonomy  and  categorize 
the  appropriate  levels  of  autonomy  when  dealing  with  a  given  core  function.  Levels  of  control  for 
the  UAS  have  been  defined,  but  this  is  not  to  be  confused  with  levels  of  autonomy  which  define 
specifically  how  a  vehicle  operates  and  its  ability  to  observe,  orient,  decide,  and  act  (OODA). 
More  specifically,  how  “in  the  loop”  does  the  vehicle  require  its  pilot  or  operator  to  be? 

Researchers  at  Air  Force  Research  Laboratory  (AFRL)  attempted  to  tackle  this  issue  in  2002 
by  defining  levels  of  autonomy,  which  they  defined  as  1)  the  ability  to  generate  one’s  own  pur¬ 
poses  without  any  instruction  from  the  outside,  and  2)  having  free  will9.  AFRL  used  three  metrics 
to  measure  the  autonomy  of  the  UAS,  graduated  on  ten  levels  -  perception/situational  awareness, 
analysis/decision  making,  and  communication/cooperation. 

Why  in  the  Loop? 

Before  delving  into  how  we  test  the  handling  qualities  of  a  pilot-in-the-loop  UAS,  we  must 
first  answer  the  question  “why  design  the  system  with  the  pilot  in  the  loop?”  The  concept  of  a 
robotic  airplane  can  immediately  conjure  up  flashbulb  memories  of  Hollywood  films  in  which 
robots  become  “self-aware,”  multi-task,  and  execute  missions  with  capabilities  far  beyond  that  of 
even  the  most  storied  of  superheroes.  Reality  dictates  that  as  of  2015,  AI  is  not  quite  that  sophis¬ 
ticated.  Further,  removal  of  pilot  from  the  airplane  leads  to  a  tendency  to  separate  UAS  from 
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manned  aircraft.  While  in  many  cases  this  is  warranted,  there  still  exist  core  functions  in  which 
pilot  in  the  loop  control  is  appropriate  and  necessary. 

Unmanned  systems  are  one  of  three  configurations:  remotely-piloted,  fully-autonomous,  or  a 
mixture  of  the  two.  The  remotely-piloted  system  uses  a  controlling  station  interfaced  with  a  pilot 
that  sends  flight  control  commands  to  the  aircraft  using  a  data  link.  A  fully-autonomous  system 
also  uses  a  controlling  station  but  instead  of  piloted  control,  a  mission  plan  is  sent  to  the  aircraft 
which  is  then  processed  by  the  onboard  flight  control  system  and  executed  without  any  further 
uplinked  commands.  The  mission  may  be  monitored  from  any  number  of  terminals  and  changes 
to  the  mission  plan  can  be  sent  from  the  control  station  via  an  operator  but  the  operator  does  not 
have  direct  control  of  the  aircraft.  This  can  also  be  thought  of  as  pilot  “on  the  loop.”  Fully  auton¬ 
omous  systems,  however,  are  limited  in  mission  scope  to  intelligence,  surveillance,  and  recon¬ 
naissance  (1SR)  missions  and  do  not  allow  for  combat  missions,  at  least  in  today’s  battlespace. 

Although  UAS  depend  on  autonomy  a  great  deal,  there  is  much  to  be  gained  by  having  a  pilot 
in  the  loop.  First,  flight  testing  of  aircraft  performance  and  flying  qualities  is  greatly  facilitated  by 
having  a  pilot  in  the  loop.  Flowever,  design  for  mission  suitable  aircraft  are  seldom,  and  should 
not  be  designed  simply  to  optimize  test  and  evaluation.  Even  higher  levels  of  autonomous  flight 
that  require  minimal  pilot  interaction  with  flight  controls  are  achieved  via  the  airplane  interacting 
with  the  four  basic  forces  of  flight,  and  must  achieve  stability  and  control  via  algorithms  which 
compensate  for  the  effects  in  that  domain.  No  feedback  loop  or  computer  algorithm  can  deliver 
the  same  qualitative  feedback  as  a  pilot. 

The  deployment  of  weapon  systems  demands  a  pilot  in  the  loop.  As  an  integral  part  of  the  kill 
chain,  it  is  ultimately  the  pilot  who  makes  the  final  decision  on  weapon  employment.  Although 
any  process  can  be  automated,  there  is  far  too  much  at  stake  when  weapon  employment  is  in¬ 
volved.  Just  as  the  decision  whether  to  take  lives  should  not  be  automated,  the  final  authority  for 
weapon  release  should  rest  with  a  human.  This  is  not  to  infer  that  humans  are  infallible,  but  acts 
of  war  demand  accountability  and  responsibility,  and  neither  can  be  delegated  to  Al.  In  the  com¬ 
mercial  sector,  missions  do  not  require  deliberate  choices  between  life  and  death;  however  the 
automation  must  execute  its  mission  at  or  better  than  the  human  can  in  all  conditions. 

Lastly,  takeoff  and  landing  are  the  most  flight  critical  portions  of  the  mission.  Between  fiscal 
year  2000  and  2013,  15%  of  MQ-U  and  MQ-9 '  Class  A  mishaps  occurred  during  the  takeoff  and 
landing  phase,  many  of  those  in  the  form  of  PIO  during  the  landing  phase10.  Due  to  this  abnor¬ 
mally  high  landing  accident  rate,  many  unmanned  systems  already  employ  automatic  takeoff  and 
landing  systems.  In  fact,  similar  systems  have  been  developed  for  manned  aircraft  These  takeoff 
and  landing  systems  (TALS)  must  work  100%  of  the  time  otherwise  the  systems  will  not  be  relia¬ 
ble  and  will  not  be  used  operationally.  The  systems  that  have  already  been  developed  for  land 
based  operations  have  proven  to  be  satisfactory  in  some  aspects.  Flowever,  as  more  sophisticated 
recovery  algorithms  continue  to  evolve,  there  will  always  be  a  need  to  understand  at  the  most 
basic  level  how  the  aircraft  gets  safely  back  to  the  runway.  In  some  cases,  pilot-in-the-loop  design 
may  help  mitigate  risk  initially.  As  systems  begin  to  integrate  with  manned  platforms  in  FAA- 
controlled  airspace,  they  will  need  to  afford  the  user  flexibility  to  see-think-act  in  a  way  that 
meets  or  exceeds  the  ability  of  the  human  pilot.  As  of  2015,  the  operation  of  an  unmanned  aircraft 
at  any  airfield  is  a  major  event.  Notices  to  Airmen  (NOTAMs)  are  issued,  Temporary  Flight  Re¬ 
strictions  (TFRs)  are  enacted,  radio  calls  are  made;  and  for  all  intents  and  purposes  everyone  op¬ 
erates  under  the  understanding  that  operations  are  anything  but  normal.  Operating  with  a  pilot  in 
the  loop,  talking  on  the  radio,  controlling  the  aircraft  is  simply  one  way  to  bridge  the  gap  for  the 
time  being. 
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Based  on  these  reasons,  it  is  important  to  maintain  a  pilot  in  the  loop  of  the  UAS,  especially 
with  the  growing  importance  of  the  UAS  as  a  military  asset  and  with  the  ever  increasing  scope  of 
the  UAS.  Since  the  pilot  is  in  the  loop  of  the  UAS  for  important  tasks  of  the  mission,  the  handling 
qualities  of  the  aircraft  need  to  be  properly  designed  and  tested. 

UAS  SYSTEM  ARCHITECTURE 

The  basic  elements  of  a  UAS  are  an  unmanned  aircraft,  a  data  link,  and  a  control  station.  The 
interactions  between  these  three  systems  make  up  the  entire  system.  When  testing  for  handling 
qualities  in  a  UAS,  it  is  important  to  understand  and  separate  the  effects  of  the  hardware,  soft¬ 
ware,  and  the  datalink.  The  hardware  component  primarily  comprises  of  the  stick  and  rudder  pe¬ 
dal  dynamics,  ergonomics  in  the  control  station,  and  the  control  surfaces  and  actuators  in  the  air¬ 
craft.  The  software  may  contain  displays,  behaviors,  or  logic  that  change  how  the  pilot  interacts 
with  the  vehicle.  Finally  the  datalink  is  the  primary  culprit  for  introducing  time  delay  and  can 
have  adverse  effects  on  the  handling  qualities  of  the  UAS. 

Control  Station 

The  control  station  is  any  device  or  system  that  supports  autonomous  or  manual  control  of  the 
RPA.  It  can  be  fixed,  mobile,  hand  held,  or  mounted  in  another  aircraft.  The  basic  functionality  of 
these  different  control  stations  is  the  same  and  therefore,  for  simplicity,  this  paper  will  only  refer 
to  Ground  Control  Stations  (GCS)  that  contain  a  basic  cockpit.  The  basic  cockpit  in  a  GCS  may 
consist  of  a  throttle,  control  stick,  and  rudder  pedals  as  well  as  any  other  hardware  inputs  benefi¬ 
cial  to  the  pilot  (landing  gear  controls,  flap  lever,  munitions  arming  switch,  engine  kill  switch, 
Hands-on  Throttle  and  Stick  [HOTAS],  etc.).  The  cockpit  also  has  a  video  feed  from  the  aircraft, 
normally  overlaid  onto  a  HUD.  The  video  feed  can  be  sourced  from  any  number  of  onboard  sen¬ 
sors  such  as  a  daylight  video  camera,  Electro-optical/Inffa-red  (EO/IR)  camera  or  a  combination 
thereof. 

To  control  the  RPA,  a  pilot  makes  an  input  into  the  system  (control  stick  or  throttle  movement 
for  example),  then  the  signal  is  processed  by  the  GCS  and  sent  to  a  data  terminal  that  transmits 
the  signal  across  the  data  link  (normally  radio  frequency  [RF]  or  satellite  communication).  The 
RPA  receives  this  data  using  its  own  data  link  antennae  and  processes  the  data  using  an  onboard 
computer  which  outputs  a  command  to  the  flight  controls,  resulting  in  the  desired  output.  An 
overview  of  the  system  is  provided  in  Figure  4. 


GROUND 

CONTROL 
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Figure  4  UAS  Overview 

With  today’s  computers,  this  process  can  be  accomplished  rather  quickly,  on  the  order  of —150 
milliseconds  (ms)  for  RF  data  links  within  Line-of-Sight  (LOS)  of  the  RPA.  For  satellite  commu¬ 
nications,  the  latency  is  approximately  1-2  seconds.  While  fast,  these  do  not  compare  with  the 
latency  of  manned  aircraft  that  exhibit  imperceptible  time  delays,  usually  less  than  5  millisec¬ 
onds,  which  is  considered  a  minimum  threshold  for  time  delay  perceptibility.  To  account  for  this 
delay,  the  flight  control  system  uses  an  attitude  command  system  instead  of  the  conventional  atti¬ 
tude  rate  system  which  will  be  discussed  in  the  flight  controls  section. 

UAS  Cockpit 

There  are  several  differences  in  the  UAS  cockpit  from  the  manned  cockpit  that  make  design¬ 
ing  a  flight  control  system  more  complex  than  a  manned  aircraft. 

As  previously  mentioned,  a  standard  UAS  cockpit  will  consist  of:  a  visual  display,  throttle, 
control  stick,  rudder  pedals,  and  other  appropriate  hardware  switches  and  levers.  Removing  the 
pilot  from  the  aircraft  reduces  the  amount  of  information  available  to  the  pilot.  Of  his  five  senses, 
the  UAS  pilot  has  only  sight  to  rely  on  for  aircraft  control.  While  taste  and  smell  are  still  affected 
by  what  other  crewmember  brought  for  lunch,  things  we  tend  to  take  for  granted  have  all  of  the 
sudden  gone  away  or  have  been  replaced  -  the  hum  of  the  engine  is  substituted  by  the  whir  of 
computers  and  servers,  the  crew  experiences  1-g  at  all  times,  there’s  no  smell  of  jet  fuel,  exhaust, 
and  freedom  -  even  vision  is  reduced  to  two-dimensions  with  no  periphery.  As  one  wise  MQ-1 
instructor  pilot  once  said  “any  resemblance  to  airplane  flying  as  you  know  it  is  purely  coinci¬ 
dental.” 
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The  pilot  control  loop  for  an  unmanned  system  is  represented  in  Figure  5  below,  modified 
from  Figure  1: 


Figure  5  UAS  Modified  Handling  Qualities  Control  Loop 

As  previously  mentioned,  the  majority  of  sensory  cues  that  are  present  in  a  manned  platform 
are  not  available  when  the  pilot  is  removed  from  the  aircraft.  For  the  MQ-1  and  MQ-9  aircraft, 
visual  cues  are  replaced  by  a  nose  camera  and  a  HUD.  The  nose  camera  can  provide  the  pilot 
with  terrain  features  and  weather  conditions  but  due  to  its  limited  Field  of  View  (FOV),  it  does 
not  provide  great  Situational  Awareness  (SA)  for  collision  avoidance  with  other  aircraft,  terrain 
avoidance  during  turns  or  dynamic  maneuvers,  or  the  best  ground  reference  for  landing.  Most 
RPA,  however,  have  a  separate  onboard  camera  attached  to  a  gimbal  and  operated  by  another 
crew  member  that  provides  (some)  missing  situational  awareness. 

The  HUD  provides  the  pilot  with  important  aircraft  parameters  such  as  airspeed,  altitude,  An¬ 
gle  of  Attack  (AoA),  pitch  attitude,  roll  attitude,  vertical  velocity,  engine  status,  etc.  Due  to  the 
limited  bandwidth  of  the  data  link,  some  of  these  parameters  may  be  sampled  at  relatively  low 
rates.  This  can  cause  the  display  to  appear  “choppy”  which  adds  another  degree  of  difficulty  for 
the  pilot  when  executing  mission  tasks.  On  the  other  end  of  the  scale,  parameters  that  have  a  high 
sample  rate  coupled  with  a  highly  sensitive  instrument  can  appear  to  fluctuate  at  high  frequencies, 
preventing  the  pilot  from  determining  the  value  of  the  parameter. 

UAS  Datalink 

In  order  to  communicate  with  the  UAS,  the  aircraft  utilizes  one  or  more  data  links.  Data  link 
communication  is  performed  over  RF  to  communicate  to  the  UAS  in  direct  LOS  or  Beyond  Line- 
of-Sight  (BLOS).  BLOS  communication  can  be  achieved  by  communicating  to  the  UAS  via  an¬ 
other  platform,  whether  it  is  a  satellite,  a  naval  ship,  or  another  airborne  platform.  The  BLOS 
communication  increases  the  amount  of  latency  in  the  system  since  the  data  has  to  travel  further 
than  LOS  and  has  to  be  processed  through  various  systems.  The  increased  latency  results  in  ad¬ 
verse  effects  on  handling  qualities  and  it  is  very  rare  for  BLOS  communications  to  be  used  for 
high  risk  handling  qualities  tasks  such  as  takeoff  and  landing.  Those  tasks  are  commonly  per¬ 
formed  using  a  LOS  datalink  after  which  a  BLOS  datalink  can  be  used  for  operations  that  are  be¬ 
yond  line  of  sight  of  the  ground  station  data  terminal. 
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The  RPA 


The  RPA  is  very  similar  to  manned  vehicles  in  almost  every  regard  with  the  exception  of 
communication  and  control.  First,  removing  the  pilot  from  the  vehicle  creates  the  requirement  for 
remote  control  of  the  aircraft  which  is  accomplished  using  a  data  link.  Since  the  aircraft  is  being 
manually  controlled  during  critical  phases  of  flight,  such  as  takeoff  and  landing,  redundancy  is 
built  into  the  communication  system  on  the  aircraft.  Antennae  are  installed  on  both  the  upper  and 
lower  section  of  the  fuselage  to  prevent  antennae  blockage  during  turning  and  both  directional 
and  omni-directional  antennae  are  used  for  additional  redundancy.  In  addition,  the  RPA  flight 
computer  has  a  built  in  logic  that  will  autonomously  fly  the  aircraft  should  the  link  between  the 
ground  station  and  the  aircraft  become  lost. 

Second,  the  latency  of  the  data  link  system  causes  handling  qualities  issues  that  are  addressed 
by  the  use  of  a  different  type  of  flight  control  system. 


UAS  Flight  Control  System 

On  any  aircraft,  the  latency  from  pilot  input  to  aircraft  response  back  to  pilot  input  is  a  large 
determining  factor  in  the  handling  qualities  of  the  vehicle.  If  there  is  a  large  latency,  the  pilot  can 
become  out  of  phase  with  the  aircraft  response  and  develop  a  PIO  which  is  undesirable  for  han¬ 
dling  qualities. 


In  a  manned  platform,  the  latency  is  from  pilot  input  to  aircraft  response,  such  as  that  caused 
by  transport  time  delay  in  a  digital  flight  control  system.  In  an  unmanned  platform,  the  aircraft 
response  has  to  be  downlinked  back  to  the  ground  control  station  for  the  pilot  to  recognize  and 
react  to  it.  This  additional  step  increases  the  equivalent  system  time  delay.  One  common  method 
used  in  RPA  to  compensate  for  time  delay  is  using  an  attitude-command  flight  control  system.  In 
an  attitude -command  system  the  RPA  flight  controls  are  commanded  by  angular  commands  in¬ 
stead  of  rate  commands.  A  pilot’s  control  stick  input  into  the  system  will  command  a  specific 
pitch  and/or  roll  angle  which  is  sent  to  the  aircraft.  When  the  aircraft  receives  that  command,  it 
uses  an  onboard  control  system  to  deflect  the  control  surfaces  based  on  current  vs  commanded 
attitude  angle.  An  example  of  such  a  control  system  is  a  Proportional-Integral-Derivative  (PID) 
controller.  This  controller  uses  a  control  loop  to  continuously  deflect  the  control  surfaces  to  re¬ 
duce  the  error  between  the  commanded  and  actual  attitude  of  the  aircraft.  Anytime  the  command¬ 
ed  angle  is  different  from  the  aircraft  angle,  the  controller  will  deflect  the  control  surface  based 
on  the  sum  of  its  proportional,  integral,  and  differential  gain  values.  An  example  PID  controller  is 
shown  in  Figure  6. 


Attitude 

Command 


Figure  6  PID  Controller 


The  proportional  term  provides  a  component  of  the  control  surface  deflection  by  multiplying 
the  proportional  gain  value  by  the  difference  between  commanded  and  actual  attitude  angle.  The 


11 


integral  term  is  the  sum  of  the  error  between  resulting  attitude  angle  and  command  over  time  mul¬ 
tiplied  by  the  integral  gain  value.  The  differential  term  is  the  slope  of  the  error  over  time  multi¬ 
plied  by  the  differential  gain  value.  Adding  all  three  terms  together  yields  the  control  surface  de¬ 
flection  value.  This  value,  however,  is  usually  limited  by  output  limits  and  output  rate  limits. 

Unlike  the  flight  control  system  on  a  manned  platform,  rapid  changes  in  stick  deflection  do 
not  always  equate  to  the  same  rate  of  aircraft  response.  The  onboard  processor  will  command 
larger  control  surface  inputs  for  larger  deltas  between  commanded  angle  and  current  angle  but 
there  is  generally  an  error  limit  to  how  large  the  delta  can  be.  For  example,  if  the  error  limit  is 
±10°  for  pitch,  an  input  of  +20°  pitch  from  a  0°  pitch  angle  will  result  in  the  processor  using  a 
delta  value  of  10°  instead  of  20°.  Additionally,  the  onboard  control  processor  normally  has  a  rate 
limiter  which  prevents  large  changes  in  attitudes  rates  from  occurring.  With  these  two  limiters,  it 
is  more  difficult  for  the  pilot  to  receive  proper  feedback  from  his  control  stick  inputs. 

Another  method  used  to  control  an  RPA  involves  commanding  roll  attitude  (ij))  with  lateral 
stick  and  flight  path  (y)  with  the  pitch  stick  in  the  longitudinal  axis.  A  simple  way  to  think  of  a 
y/())-command  flight  control  system  is  as  a  shortcut  to  achieve  a  desired  effect.  The  pilot  controls 
flight  path  directly  using  the  Flight  Path  Marker  (FPM)  to  control  the  velocity  vector  in  the  longi¬ 
tudinal  axis,  and  controls  roll  using  lateral  stick  to  command  desired  hank  angle.  Throttle  controls 
airspeed  for  front-side  of  the  power  curve  maneuvering.  The  pilot’s  task  is  reduced  to  pointing  the 
aircraft  to  an  end  state  and  the  aircraft  responds,  essentially  freeing  the  brain  from  the  task  of  us¬ 
ing  angles  or  other  parameters  to  model  and  effect  flight  path. 

Recent  research  at  the  Air  Force  Test  Pilot  School  has  even  used  a  similar  principle  to  employ 
predictive  cueing  to  compensate  for  time  delays  up  to  2  seconds.  Projects  HAVE  PIRATE  and 
HAVE  ORACLE  experimented  with  lead-compensation  algorithms  on  a  surrogate  variable- 
stability  Learjet  (SVS)  and  demonstrated  the  ability  to  control  the  VSS  with  considerable  time 
delays.  Although  the  closed  loop  control  systems  modeled  were  attitude  command,  the  lead  com¬ 
pensators  used  predictive  FPMs  which  represented  the  vehicle’s  velocity  vector  up  to  two  sec¬ 
onds  in  advance  of  the  feedback  image  on  the  HUD.  HAVE  ORACLE  demonstrated  Level  1 
handling  qualities  in  a  closed-loop  attitude  command  flight  control  system  with  up  to  two  seconds 
of  latency  for  tasks  at  altitude  and  Level  2  for  landing  tasks.  Handling  qualities  degradations  in 
the  landing  phase  were  attributed  to  incorrect  modeling  of  ground  effect  by  the  predictive  cueing 
algorithm11. 

HANDLING  QUALITIES  EVALUATIONS  OF  UNMANNED  SYSTEMS 

Handling  qualities  must  be  evaluated  for  UAS  that  require  pilot-in-the-loop  control  to  some 
degree.  At  low  levels  of  automation,  it  is  appropriate  to  use  similar  methods  to  evaluate  handling 
qualities  as  manned  aircraft  while  recognizing  and  testing  to  the  unique  attributes  of  the  entire 
UAS  -  the  aircraft,  the  datalink,  and  the  control  station.  For  systems  that  have  higher  levels  of 
automation,  it  may  not  be  appropriate  to  apply  principles  developed  on  manned  aircraft.  A  num¬ 
ber  of  other  methods  have  been  developed  and  researched  for  highly  automated  systems;  howev¬ 
er,  they  will  not  be  discussed  in  the  following  sections. 

In  order  to  design  a  handling  qualities  evaluation  for  a  UAS,  it  is  important  to  understand  how 
a  pilot  interacts  with  the  vehicle,  processes  information,  and  compensates  for  undesirable  mo¬ 
tions.  Two  factors  which  induce  undesirable  motions  are  time  delay  and  stick  sensitivity.  For  a 
UAS,  time  delay  is  inevitable  and  often  the  question  is  “how  much  time  delay  is  there?”  Stick 
sensitivity  causes  problems  in  an  attitude  command  flight  control  system  because  stick  deflection 
equals  pitch  or  roll  attitude,  and  an  over-sensitive  control  stick  may  make  precise  inputs  difficult. 
Often,  UAS  control  stations  use  Commercial-off-the-Shelf  (COTS)  hardware  with  little  influence 
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over  factors  such  as  stick  deflection,  spring  tension,  and  control  harmony.  Additionally,  HOTAS 
functionality  tends  to  drive  the  design  with  handling  qualities  being  an  afterthought. 

Test  Approach 

For  this  discussion  we  examine  a  three-phased  approach  to  conducting  a  handling  qualities 
evaluation  that  builds  from  low  gain  maneuvering  to  high  gain  maneuvering  to  an  operationally 
representative  task  that  yields  a  Cooper-Harper  rating  and  an  insight  on  whether  the  aircraft  is 
Level  1,  Level  2,  or  Level  3. 

The  first  phase  can  be  characterized  by  low  gain,  low  bandwidth  maneuvering.  In  some  ways 
Phase  1  can  be  thought  of  as  a  warm  up  in  which  the  pilot  can  qualitatively  assess  the  aircraft 
handling  in  a  docile  environment.  In  this  phase,  the  pilot  can  get  a  first  look  at  time  delay,  stick 
sensitivity,  and  control  harmony  and  be  reasonably  confident  that  he  can  maintain  control  while 
observing  the  bare  airframe  response.  In  UAVs,  the  flight  control  systems  are  closed  loop,  and 
evaluating  bare  airframe  would  be  impossible  and  impractical.  Thus,  Phase  1  maneuvering  would 
still  be  accomplished  to  evaluate  the  augmented  airframe  response.  Pilots  fly  open-loop  maneu¬ 
vers  such  as  pulses,  steps,  doublets,  and  bank-to-bank  rolls  in  this  phase.  Additionally,  quasi-open 
loop  maneuvers  can  be  flown  such  as  pitch  and  heading  captures.  Phase  1  may  take  all  of  a  few 
minutes  or  as  long  as  necessary  until  the  test  team  is  comfortable  proceeding  to  the  next  phase. 
For  a  UAS,  Phase  1  will  include  an  evaluation  of  displays  that  provide  control  and  performance 
information  for  the  aircraft  flown.  A  pilot  may  have  the  opportunity  to  fly  the  aircraft  while  look¬ 
ing  through  a  number  of  video  sources,  if  available.  Data  obtained  in  this  phase  is  primarily  quali¬ 
tative,  but  quantitative  data  can  be  used  to  compare  to  models  and  validate  the  flight  control  sys¬ 
tem. 

Next,  Phase  2  maneuvering  seeks  to  explore  high-gain,  high-bandwidth  control  of  the  aircraft. 
The  primary  objective  here  is  to  seek  out  and  identify  handling  qualities  cliffs  in  which  control 
appears  nominal  to  a  certain  extent  then  rapidly  degrades  into  a  PIO  situation.  Phase  2  evaluations 
are  conducted  at  a  safe  altitude  which  reduces  risk  if  the  pilot  encounters  PIO  by  allowing  the 
aircrew  time  to  recover  the  aircraft  in  case  of  unexpected  behavior.  Many  avenues  exist  to 
achieve  this  goal;  however  the  lack  of  senses  while  flying  a  UAS  mentioned  earlier  requires  some 
degree  of  creativity  on  the  part  of  the  test  team  for  planning  and  execution.  One  technique  used  in 
manned  aircraft  testing  is  Handling  Qualities  during  Tracking  (HQDT)  that  involves  a  target  air¬ 
craft  ahead  of  the  test  aircraft  that  maneuvers  randomly  for  the  test  aircraft  to  track.  The  test  air¬ 
craft  then  uses  high  frequency,  high  amplitude  inputs  to  aim  for  zero-error  tracking.  During  this 
technique  the  pilot  must  resist  the  urge  to  compensate  by  making  low-gain  inputs  as  the  puipose 
is  to  explore  the  entire  envelope  for  maneuvering.  This  technique  is  impractical  for  UAS  because 
the  camera  FOV  is  limited  and  judging  distance  from  the  target  aircraft  is  very  difficult.  Picking  a 
fixed  point  in  space  about  which  to  maneuver  also  lends  problems  since  over  time  the  pilot  will 
inevitably  dwell  to  zero  error  and  may  never  elevate  gains  enough  to  meet  success  criteria  for  the 
test  point.  One  method  to  elevate  gains  without  dwell  is  to  use  the  pitch  ladders  on  the  HUD 
combined  with  the  “W”  marker,  which  indicates  feedback  pitch  attitude.  Using  the  pitch  ladders, 
establish  two  zones  UP  and  DOWN,  which  are  between  0°  and  5°  pitch,  positive  and  negative,  as 
shown  in  Figure  7.  This  is  called  a  Workload  Buildup/Boundary  Avoidance  Tracking 
(WLB/BAT)  task. 
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Figure  7  HUD  Boundaries  for  WLB/BAT  in  the  Pitch  Axis 

Begin  the  maneuver  in  level  flight  then  with  low  frequency  inputs  maneuver  to  either  the  UP 
or  DOWN  boundary,  as  called  by  the  test  conductor,  who  stands  behind  the  pilot  and  directs  the 
test  point.  As  the  maneuver  continues,  the  pilot  will  build  up  in  workload  by  making  higher  fre¬ 
quency  inputs  and  the  test  conductor  will  call  for  the  pilot  to  switch  boundaries  at  irregular  inter¬ 
vals.  If  the  pilot  is  able  to  maintain  the  boundary  with  elevated  gains,  recover  to  level  flight  and 
establish  smaller  boundaries,  say  two  and  a  half  degrees  wide  as  shown  above,  then  repeat  the 
WLB/BAT  task  to  the  smallest  boundary  or  zero  error  tracking.  If  at  any  point  the  pilot  encoun¬ 
ters  PIO,  call  knock-it-off  and  recover  to  level  flight.  For  an  attitude  command  flight  control  sys¬ 
tem,  all  the  pilot  needs  to  do  is  neutralize  the  stick  and  the  aircraft  will  return  to  a  level,  trimmed 
condition.  This  same  task  can  be  performed  in  the  roll  axis  using  the  roll  bars  on  the  HUD,  as 
shown  in  Figure  8. 


Figure  8  HUD  Boundaries  for  WLB/BAT  in  the  Roll  Axis 

After  performing  WLB/BAT  tasks,  the  pilot  will  assign  a  PIO  rating  to  the  task,  which  assess¬ 
es  the  PIO  susceptibility  of  the  aircraft  when  performing  the  task.  The  PIO  rating  scale,  shown  in 
Figure  9,  is  used  to  evaluate  PIO  tendencies  and  can  be  used  with  both  Phase  2  maneuvering  tasks 
and  Cooper-Harper  tasks12,  which  will  be  discussed  in  Phase  3.  At  first  glance  it  may  seem  coun¬ 
terintuitive  to  fly  an  aircraft  in  the  manner  described  above.  The  reason  for  this  is  to  explore  the 
edges  of  the  envelope  in  which  the  aircraft  may  be  controlled,  from  the  most  seasoned  pilot  to  the 
greenest  novice. 
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DECISIONS 


DESCRIPTIONS 


RATINGS 


- >1  No  tendency  for  pilot  to  induce  undesirable  motions. 

E 3 

No 

Undesirable  motions  tend  to  occur  when  pilot  initiates  abrupt  maneuvers 
or  attempts  tight  control.  These  motions  can  be  prevented  or  eliminated 
by  pilot  technique 

Undesirable  motions  easily  induced  when  pilot  initiates  abrupt  maneuvers 
or  attempts  tight  control.  These  motions  can  be  prevented  or  eliminated 
but  only  at  sacrifice  to  task  performance  or  through  considerable  pilot 
attention  and  effort. 


Oscillations  tend  to  develop  when  pilot  initiates  abrupt  maneuvers  or 
attempts  tight  control.  Pilot  must  reduce  gam  or  abandon  task  to  recover. 


U 


Yes 

- > 

Divergent  oscillations  tend  to  develop  when  pilot  initiates  abrupt 
maneuvers  or  attempts  tight  control.  Pilot  must  open  loop  by  releasing  or 
freezing  the  stick. 

Disturbance  or  normal  pilot  control  may  cause  divergent  oscillation.  Pilot| 
must  open  control  loop  by  releasing  or  freezing  the  stick. 


Figure  9  PIO  Rating  Scale 

The  final  piece,  or  Phase  3,  of  a  handling  qualities  evaluation  is  the  operational  evaluation  us¬ 
ing  the  Cooper-Harper  rating  scale  shown  in  Figure  2.  Desired  and  adequate  performance  criteria 
are  assigned  to  a  given  task.  Performance  is  assessed  as  well  as  workload  and  pilot  compensation 
in  order  to  yield  a  CHR.  The  importance  of  properly  building  up  to  this  point  cannot  be  over 
stressed.  With  a  flight  controller  that  has  not  been  previously  characterized,  it  is  not  advisable  to 
jump  right  into  a  Cooper-Harper  task  without  evaluating  low-gain,  low-bandwidth  maneuvering 
and  checking  for  cliffs  first. 

Phase  3  tasks  use  operationally  representative  tasks  to  determine  the  handling  qualities  of  the 
system.  For  manned  systems,  these  tasks  can  vary  from  takeoffs,  landings,  formation  with  other 
aircraft,  air  to  air  tracking,  aerial  refueling,  bombing  runs,  to  whatever  the  aircraft’s  mission  de¬ 
mands.  For  each  of  these  tasks,  the  pilot  is  constantly  updating  his  control  inputs  based  on  desired 
aircraft  response  and  actual  aircraft  response.  Tasks  such  as  air-to-air  tracking,  formation  flight, 
and  aerial  refueling  are  specifically  difficult  for  the  pilot  because  the  target  is  moving  and  not 
always  stable.  Additionally  when  in  close  proximity  to  another  aircraft,  a  pilot’s  gains  are  natural¬ 
ly  elevated  due  to  the  consequence  of  a  midair  collision.  This  requires  the  pilot  to  maintain  very 
tight  control  of  the  aircraft.  These  tasks  are  great  for  Phase  3  because  they  keep  the  pilot  in  the 
loop  with  the  system,  require  tight  control  of  the  aircraft,  and  require  an  elevated  pilot  workload. 
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Evaluating  Unmanned  Systems 

To  determine  operational  tasks  in  the  context  of  a  UAS  handling  qualities  evaluation,  first  de¬ 
fine  the  aircraft’s  mission.  Group  4  and  5  UAS  are  medium-to-high  altitude  long-endurance  plat¬ 
forms,  some  with  ability  to  employ  weapons.  These  platforms  can  execute  missions  including  but 
not  limited  to  Intelligence,  Surveillance  and  Reconnaissance  (1SR),  Strike  Coordination  and  Re¬ 
connaissance  (SCAR),  Combat  Search  and  Rescue  (CSAR),  Close  Air  Support  (CAS),  Air  Inter¬ 
diction  (Al),  and  Dynamic  Targeting/Time-Sensitive  Targeting  (DT/TST)13.  Operational  tasks 
associated  with  these  missions  include  takeoffs,  landings,  surveillance  of  stationary  and/or  mov¬ 
ing  ground  targets,  and  weapon  deployment  on  stationary  and/or  moving  targets.  Operationally, 
UAS  have  not  been  required  to  perform  aerial  refueling,  formation  flight  with  another  aircraft,  or 
air  to  air  weapons  employment.  Handling  qualities  evaluations  are  only  useful  for  tasks  the  air¬ 
craft  has  been  designed  to  perform,  therefore,  testing  must  focus  on  those  operational  tasks  that 
keep  the  pilot  in  the  loop  with  the  aircraft,  require  some  degree  of  tight  control  of  the  aircraft,  and 
require  an  elevated  pilot  workload.  It  is  desirable  for  an  operational  task  to  meet  all  three  criteria. 

By  design,  UAS  have  highly  automated  flight  control  logic  which  is  beneficial  for  aircraft 
control  but  requires  extra  care  when  designing  a  handling  qualities  evaluation.  Therefore  the  tasks 
must  be  designed  to  avoid  scenarios  where  the  performance  of  the  task  is  measured  by  the 
onboard  controller  and  not  by  the  pilot  inputs.  For  instance,  one  operationally  representative  task 
for  the  UAS  is  takeoff.  Assume  the  task  of  the  takeoff  is  to  capture  a  1 0  degree  pitch  attitude  after 
rotation  and  maintain  that  pitch  attitude  until  1,000  feet  Above  Ground  Level  (AGL).  At  rotation 
speed,  the  pilot  will  move  the  stick  to  a  position  that  indicates  a  1 0  degree  pitch  command.  The 
aircraft  flight  control  system  will  then  maneuver  the  aircraft  to  this  attitude  and  continuously  ma¬ 
nipulate  the  control  surfaces  to  maintain  the  attitude  during  climb  out.  This  specific  task  only 
measures  the  pilot-vehicle  system  to  the  point  of  capturing  desired  pitch  after  which  it  measures 
the  aircraft’s  ability  to  maintain  a  pitch  command,  unless  of  course  stick  sensitivity  makes  it  im¬ 
possible  to  capture  and  maintain  a  pitch  attitude.  A  better  task  would  be  to  capture  an  initial  pitch 
attitude  at  rotation  then  maintain  a  climb  speed  after  rotation  (±2  KLAS)  until  1000  feet  (ft)  AGL. 
In  this  scenario,  the  pilot  would  pull  the  stick  back  after  rotation,  accelerate  to  the  target  climb 
speed,  then  continuously  adjust  throttle  and/or  pitch  attitude  to  maintain  that  airspeed  during  the 
climb.  Since  this  task  requires  the  pilot  to  constantly  update  his/her  inputs  into  the  system,  it 
keeps  the  pilot  in  the  loop  with  the  aircraft. 

To  generate  tight  control  of  the  aircraft,  the  task  should  require  the  pilot  to  track  an  item  or  pa¬ 
rameter  that  is  constantly  changing.  This  will  require  the  pilot  continually  update  his  or  her  input 
to  the  flight  control  system  and  could  uncover  issues  with  latency  and/or  control  system  gain  and 
phase  margins.  An  example  of  such  a  task  where  the  item  the  pilot  is  tracking  is  constantly  chang¬ 
ing  is  tracking  a  run  in  line  for  a  weapons  deployment.  The  pilot’s  HUD  has  a  Flight  Path  Marker 
(FPM)  that  uses  aircraft  velocities  to  determine  where  the  aircraft  velocity  vector  is  pointed.  For 
weapons  employment,  the  FPM  is  used  to  allow  the  pilot  to  approach  the  target  from  a  known 
direction  to  ensure  the  bomb  can  be  delivered  on  target.  The  FPM  as  shown  in  Figure  10  can  be 
moved  laterally  onto  the  run-in  line  by  adjusting  the  roll  angle  and/or  yaw  angle  of  the  aircraft. 
The  run-in  line  acts  as  a  fixed  bearing  line  from  the  target.  Since  it  is  a  fixed  line,  the  pilot  has  to 
constantly  update  the  aircraft  path  of  travel  by  making  roll  and  yaw  inputs.  Having  a  changing 
item  for  the  pilot  to  track  requires  the  pilot  to  maintain  tight  control  of  the  aircraft.  Operationally, 
this  task  is  usually  accomplished  over  BLOS  datalink  with  significant  time  delay.  Thus  this  task 
should  be  performed  in  both  LOS  and  BLOS  configurations  in  order  to  provide  an  accurate  pic¬ 
ture  of  aircraft  handling  qualities. 
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Figure  10  FPM  for  Bombing  Run 

Tying  in  all  three  factors,  operational  tasks  can  be  setup  to  properly  test  the  handling  qualities 
of  the  system.  Example  operational  tasks  for  attitude  command  flight  control  systems  are  provid¬ 
ed  in  Table  3  below. 


Table  3  UAS  Handling  Qualities  Tasks 


Task 

Suggested  Performance  Criteria 
(Desired/Adequate) 

Comments 

Takeoff 

•  Pitch  angle  at  rotation  (±0. 5/2.0) 

•  Airspeed  during  climb  to  1 ,000  ft  AGL 
(±2  K1AS/3  KIAS) 

•  Runway  tracking  during  climb  to  1,000  ft 
AGL  (±275°) 

Avoid  using  pitch  angle  through¬ 
out  takeoff  for  attitude  command 
systems.  Targeting  airspeed  tasks 
the  pilot  to  use  both  pitch  and/or 
throttle 

Normal  Landing 

•  Airspeed  during  landing  approach  (±  2 
KIAS  /4  KIAS) 

•  Glideslope  relative  to  GLS  (one-half  dot 
/  1  dot) 

•  Lateral  dispersion  from  runway  center- 
line  at  touchdown  (6  feet  /  20  feet) 

Not  advisable  to  perform  a  “spot” 
landing  since  peripheral  cues  are 
not  available  and  may  risk  driving 
nose  wheel  into  the  ground  first. 

Lateral  Offset 
Landing 

•  Same  as  normal  landing 

Lateral  offset  increases  pilot  work¬ 
load  to  correct  for  offset 

Vertical  Offset 
Landing 

•  Same  as  normal  landing 

Vertical  offset  increases  pilot 
workload  to  correct  for  offset 

Surveillance  of 
Moving  Target 

•  Depression  angle  of  video  camera  (60- 
80°  /  45-60°) 

Gimbaled  video  systems  normally 
have  limitations  on  depression  an¬ 
gle  so  the  pilot  must  judge  lateral 
distance,  vertical  distance,  and  roll 
angle  to  stay  within  camera  limits 

Weapon  De¬ 
ployment  (using 
run-in  line) 

•  FPM  tracking  to  run-in  line  (within  FPM 
circle  /  within  FPM  wings) 

Pilot  must  attempt  to  keep  the 

FPM  on  the  run-in  line  from  track¬ 
ing  to  release  of  weapon 

Design  Methods  to  Improve  Handling  Qualities  of  Unmanned  Aircraft 

We  have  just  spent  some  time  talking  about  how  to  perform  a  proper  handling  qualities  eval¬ 
uation,  culminating  in  a  rating  which  assesses  how  well  the  flight  controller  was  designed  to  ac¬ 
complish  a  given  mission.  In  this  section  we  will  discuss  design  considerations  to  improve  han¬ 
dling  qualities.  As  previously  mentioned,  stick  sensitivity  produces  undesirable  effects  for  UAS 
handling  qualities.  Stick  sensitivity  is  an  easy  problem  to  solve  if  considered  early  in  the  design 
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of  the  control  station.  Software  design  should  provide  ability  to  program  different  dead  band,  roll, 
and  pitch  angle  commands  per  stick  deflection.  In  addition  to  software,  the  flight  control  hard¬ 
ware  should  have  some  allowances  for  breakout  force  and  spring  tension  adjustment.  Each  of 
these  parameters  can  be  tested  in  a  Software  Integration  Lab  (S1L)  or  Hardware  in  the  Loop 
(HITL)  facility  at  relatively  low  cost.  Additionally,  these  should  be  tested  early  in  the  design  pro¬ 
cess  to  assist  in  eliminating  potential  handling  qualities  issues  that  may  be  encountered  later  dur¬ 
ing  flight  test,  and  will  be  exponentially  more  expensive  to  fix. 

Other  ways  to  improve  handling  qualities  include  improved  visual  cues,  lead  compensators, 
robust  designs  for  flight  critical  hardware,  and  improved  trim  systems. 

The  pilot’s  loss  of  sensory  cues  when  flying  a  UAS  requires  sole  reliance  on  visual  cues  to 
perform  flight  critical  functions  such  as  takeoff  and  landing.  Video  cameras  that  provide  a  visual 
display  for  the  pilot  are  generally  limited  in  field  of  view,  restricting  peripheral  vision.  Determin¬ 
ing  height  above  ground  becomes  more  difficult  especially  lacking  feel  for  ground  effect,  hearing 
the  engine  roll  back  to  idle,  or  sense  rate  of  touchdown.  Even  sink  rate  prior  to  touchdown  is  dif¬ 
ficult  to  judge  -  ground  rush  through  a  narrow  field  of  view  usually  appears  quite  a  bit  faster  than 
with  peripheral  vision.  To  aid  in  compensating  for  this,  UAS  designers  can  add  a  laser  altimeter 
to  help  reduce  landing  mishaps.  The  laser  altimeter  provides  the  pilot  with  an  accurate  measure¬ 
ment  for  height  above  ground  so  the  pilot  can  better  determine  when  to  flare  and  when  the  aircraft 
is  on  the  ground.  Another  improvement  that  can  be  made  to  the  system  is  to  provide  peripheral 
vision  by  adding  side  displays  in  the  cockpit  next  to  the  main  forward  facing  video  feed.  These 
side  cameras,  fed  by  either  additional  cameras  on  the  aircraft  or  by  a  graphics  simulator,  can  pro¬ 
vide  the  pilot  additional  visual  cues  to  determine  distance  from  the  ground.  If  peripheral  graphics 
are  to  be  used,  they  must  blend  exactly  with  the  nose  camera  FOV  and  display  visual  information 
at  the  same  framerate  as  the  picture  being  displayed  through  the  nose  camera;  otherwise  they  may 
become  distracting  and  degrade  handling  qualities. 

Visual  cues  can  also  take  the  form  of  lead  compensators,  which  give  the  pilot  trend  infor¬ 
mation  of  what  parameters,  either  input  or  output  will  be  in  the  future.  For  the  attitude  command 
system  already  described,  an  example  of  this  is  a  pitch  command  marker.  Since  aircraft  pitch, 
even  without  time  delay,  will  not  instantaneously  reach  a  steady  state,  an  aircraft  pitch  command 
marker  can  be  very  useful  in  providing  lead  compensation  information  to  the  pilot  to  close  the 
loop  around  pitch.  In  critical  phases  of  flight,  such  as  takeoff  and  landing,  having  this  information 
frees  up  bandwidth  for  the  pilot  to  perform  the  task  more  precisely. 

Trim  systems  for  attitude  command  flight  control  systems  allow  the  aircraft  to  fly  at  a  given 
attitude  angle  even  when  the  control  stick  is  in  the  neutral  position.  For  instance,  if  the  pilot  wants 
the  aircraft  to  climb  at  a  5°  pitch  angle,  the  pilot  can  trim  in  5°  and  center  the  stick  to  the  neutral 
position  allowing  the  aircraft  to  maintain  the  commanded  5°  pitch  trim.  This  reduces  pilot  work¬ 
load  considerably  such  that  the  pilot  need  not  hold  forces  on  the  stick  for  a  given  attitude.  During 
landing,  the  pilot  positions  the  aircraft  on  a  target  glideslope  (normally  3°  from  the  desired  touch¬ 
down  point)  and  maintains  the  glideslope  until  the  roundout  and  flare.  Without  the  trim  system, 
the  pilot  would  have  to  maintain  forward  stick  inputs  to  keep  the  aircraft  in  a  negative  pitch  angle 
that  corresponds  to  the  target  glideslope.  The  pilot  would  then  have  to  move  the  stick  from  a  for¬ 
ward  position  to  an  aft  position  during  the  flare,  through  the  stick’s  dead  band.  This  transition 
through  the  neutral  position  can  lead  to  undesired  inputs  since  most  control  sticks  are  designed 
with  some  amount  of  breakout  force  required  to  move  the  stick  out  of  the  neutral  position.  There¬ 
fore,  using  the  trim  system,  the  pilot  can  trim  in  a  negative  pitch  angle  corresponding  to  the  de¬ 
sired  glideslope  then  the  pilot  only  needs  to  apply  aft  stick  from  the  neutral  position  to  accom¬ 
plish  the  flare,  making  this  action  similar  to  how  one  would  flare  a  manned  aircraft. 
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Further  improvements  can  be  made  to  the  trim  system  to  allow  finer,  small  amplitude  changes 
to  the  trim  without  moving  the  control  stick  by  using  a  trim  “hat”  switch,  as  seen  on  most  modem 
manned  aircraft  control  sticks.  With  the  hat  switch,  making  finer  trim  adjustments  with  the  thumb 
is  more  precise  than  moving  the  entire  stick,  especially  for  pitch  adjustments  less  than  one  degree. 
Although  most  pilots  are  taught  early  in  training  not  to  fly  the  aircraft  with  trim,  for  an  attitude- 
command  system  it  makes  sense  and  can  reduce  workload  considerably.  A  common  technique  for 
flying  an  approach  to  landing  is  to  trim  2.5-3°  nose  low  and  push  or  pull  as  required  to  maintain 
glidepath  and  airspeed  until  the  flare.  This  assists  the  pilot  in  having  only  a  small  amount  of  aft 
stick  pull  required  for  the  flare,  resulting  in  a  smoother  roundout.  The  downside  to  this  is  that  the 
pilot  spends  much  of  his  time  on  approach  flying  the  aircraft  out  of  the  trimmed  condition,  which 
increases  workload.  If  the  hat  switch  were  available  to  make  finer  pitch  adjustments  while  keep¬ 
ing  the  stick  neutral,  only  a  minimal  workload  would  be  required  until  the  roundout  and  flare,  at 
which  point  the  stick  would  be  neutralized  and  more  mental  and  physical  capacity  would  be 
available  to  the  pilot  for  timing  the  roundout  and  flare.  The  MQ-9  Advanced  Cockpit  GCS  test 
pilots  practiced  this  technique  in  the  simulator  and  found  it  to  be  very  reliable. 

SUMMARY  AND  CONCLUSION 

Handling  qualities  testing  has  been  instrumental  throughout  the  history  of  aviation  in  identify¬ 
ing  deficiencies  in  piloted  control.  The  use  and  popularity  of  unmanned  systems  has  created  a 
substantial  growth  of  UAS  in  the  military  and  commercial  sector.  Although  automation  is  ever 
increasing  on  these  systems,  there  are  instances  which  require  and  will  continue  to  require  pilot  in 
the  loop  control.  Common  aspects  of  the  UAS  mission  still  require  pilot-in-the-loop  control  such 
as  takeoff,  landing,  and  weapon  deployment.  Therefore,  it  is  necessary  to  evaluate  the  handling 
qualities  of  these  systems  to  identify  any  difficulties  in  control  whether  in  normal  operation  or  in 
“comer  cases”.  Flight  test  methods  continue  to  evolve  to  evaluate  handling  qualities  in  harmony 
with  the  tried  and  true  methods  used  in  past  flight  test  programs.  UAS  have  unique  considerations 
which  must  be  accounted  for  in  evaluating  handling  qualities.  Similarly,  if  the  level  of  autonomy 
is  appropriate  for  a  given  task,  manned  aircraft  test  techniques  can  be  adapted  to  evaluate  han¬ 
dling  qualities  in  unmanned  systems. 

UAS  use  an  external,  stationary  controlling  station  which  removes  the  pilot  from  the  aircraft 
thus  removing  most  sensory  cues  from  the  pilot,  namely  proprioceptive,  aural,  and  visual.  Cou¬ 
pled  with  datalink  delays  and  display  configurations,  it  is  essential  to  understand  what  methods 
are  available  to  compensate  for  stolen  cues.  Particularly,  consideration  must  be  given  to  the  an¬ 
gle-command  flight  control  system  under  test,  whether  0  or  y,  and  how  a  task  should  be  designed 
to  evaluate  it.  Angle-command  flight  control  systems  have  been  proven  to  improve  handling  qual¬ 
ities  in  unmanned  aircraft  by  mitigating  some  of  the  effects  of  datalink  delay;  however,  there  are 
other  means  to  improve  handling  qualities  in  unmanned  aircraft.  Some  of  these  involve  lead  com¬ 
pensators  and  improved  trim  mechanisms.  Consideration  to  designing  handling  qualities  should 
be  given  early  in  the  design  process  and  often  in  the  test  process. 
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APPENDIX  A:  ACRONYMS 


AFRL 

Air  Force  Research  Laboratory 

AGL 

Above  Ground  Level 

AoA 

Angle  of  Attack 

AUVSI 

Association  for  Unmanned  Vehicle  Systems  International 

BLOS 

Beyond  Line-of-Sight 

CHR 

Cooper-Harper  Rating 

EO 

Electro-optical 

FOV 

Field  of  View 

FPM 

Flight  Path  Marker 

ft 

Feet 

GCS 

Ground  Control  Station 

GLS 

GPS  Landing  System 

HDD 

Heads  Down  Display 

HUD 

Heads  Up  Display 

IR 

Infrared 

ISR 

Intelligence,  Surveillance,  Reconnaissance 

LOS 

Line-of-Sight 

ms 

Millisecond 

OODA 

Observe,  Orient,  Decide,  Act 

PID 

Proportional-Integral-Derivative 

PIO 

Pilot-in-the-loop  Oscillation 

PVI 

Pilot  Vehicle  Interface 

RF 

Radio  Frequency 

RPA 

Remotely  Piloted  Aircraft 

SA 

Situational  Awareness 

SUAS 

Small  UAS 

SVS 

Surrogate  Variable-Stability 

TAL 

Takeoff  and  Landing  System 

TD 

Test  Director 

UAS 

Unmanned  Aircraft  System 
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